Papers with human language
VenusFactory: An Integrated System for Protein Engineering with Data Retrieval and Language Model Fine-Tuning (2025.acl-demo)
Copied to clipboard
Yang Tan, Chen Liu, Jingyuan Gao, Wu Banghao, Mingchen Li, Ruilin Wang, Lingrong Zhang, Huiqun Yu, Guisheng Fan, Liang Hong, Bingxin Zhou
| Challenge: | Pre-trained protein language models have been used in protein engineering, but their adoption is limited due to data collection, task benchmarking, and application challenges. |
| Approach: | They propose a versatile engine that integrates biological data retrieval, standardized task benchmarking, and modular fine-tuning of PLMs. |
| Outcome: | The proposed engine integrates biological data retrieval, task benchmarking, and modular fine-tuning of PLMs. |
XferBench: a Data-Driven Benchmark for Emergent Language (2024.naacl-long)
Copied to clipboard
| Challenge: | Existing methods to teach models to "language" are full of bias, toxicity, and potential intellectual property violations. |
| Approach: | They propose a benchmark for evaluating the overall quality of emergent languages using data-driven methods. |
| Outcome: | The proposed benchmark is based on utterances from the emergent language and is validated using human, synthetic, and emergentic language baselines. |
On Efficiently Representing Regular Languages as RNNs (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent work by Hewitt et al. (2020) provides an interpretation of the empirical success of recurrent neural networks (RNNs) as language models (LMs). |
| Approach: | They generalize their construction and show that RNNs can efficiently represent a larger class of LMs than previously claimed. |
| Outcome: | The results suggest that RNNs can represent a larger class of LMs than previously claimed . |
Ensemble of MRR and NDCG models for Visual Dialog (2021.naacl-main)
Copied to clipboard
| Challenge: | BLEU scores favor correct syntax over semantics. |
| Approach: | They propose a non-parametric ranking method that integrates the ranks of two strong MRR and NDCG models into a single ranking that excels on both metrics. |
| Outcome: | The proposed model can keep the MRR and NDCG models state-of-the-art and the NDGC models state of the art. |
Automatic Poetry Generation with Mutual Reinforcement Learning (D18-1)
Copied to clipboard
| Challenge: | Existing models for automatic poetry generation are based on maximum likelihood estimation (MLE) MLE-based models tend to remember common patterns of the poetry corpus, which results in loss-evaluation mismatch. |
| Approach: | They propose to model the criteria and use them as explicit rewards to guide gradient update by reinforcement learning to motivate the model to pursue higher scores. |
| Outcome: | The proposed model outperforms the current state-of-the-art model and improves on Chinese poetry. |
From Prejudice to Parity: A New Approach to Debiasing Large Language Model Word Embeddings (2025.coling-main)
Copied to clipboard
| Challenge: | Existing work in this field has looked most commonly into gender bias, racial bias, and religious bias. |
| Approach: | They propose an algorithm that uses a neural network to perform ‘soft debiasing’ and build on the seminal work of (CITATION) and (CitATION). |
| Outcome: | The proposed algorithm outperforms current methods on gender, race, and religion metrics on a wide range of metrics. |
Inducing Transformer’s Compositional Generalization Ability via Auxiliary Sequence Prediction Tasks (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing neural models lack systematic compositionality in learning symbolic structures . existing models lack this ability in learning symbols, despite being able to understand complex structures. |
| Approach: | They propose to use auxiliary sequence prediction tasks to train a Transformer model to understand compositional symbolic structures of input data. |
| Outcome: | The proposed model improves on the SCAN compositionality challenge, with only 418 (5%) training instances, and achieves 97.8% accuracy on the MCD1 split. |
Implicit Representations of Grammaticality in Language Models (2026.acl-long)
Copied to clipboard
| Challenge: | Pretrained language models generate grammatically well-formed text and discriminate well between grammatical and ungrammatically sentences in tightly controlled minimal pairs. |
| Approach: | They propose a method to train pretrained LMs for representations of grammaticality by applying perturbations to a naturalistic text corpus. |
| Outcome: | The proposed model outperforms probability-based models on human-curated grammaticality judgment benchmarks and performs worse than string probabilities on plausibility benchmarks. |
LongInsightBench: A Comprehensive Benchmark for Evaluating Omni-Modal Models on Human-Centric Long-Video Understanding. (2026.findings-acl)
Copied to clipboard
| Challenge: | LongInsightBench is the first benchmark designed to assess models’ ability to understand long videos, with a focus on human language, viewpoints, actions, and other contextual elements. |
| Approach: | They propose a benchmark to assess models’ ability to understand long videos with a focus on human language, viewpoints, actions, and other contextual elements. |
| Outcome: | The proposed model excels in three key areas: a) long-duration, human-centric videos; b) diversifying and challenging task scenarios; c) quality assurance pipeline; and d) reliability. |
Searching for the Most Human-like Emergent Language (2025.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on emergent communication systems to generate languages with high statistical similarity to human languages has not been done. |
| Approach: | They propose to optimize a signalling game-based emergent communication environment to generate state-of-the-art emergentic languages with a high degree of similarity to human language. |
| Outcome: | The proposed language generates state-of-the-art on XferBench benchmark, demonstrating its similarity to human language and entropy-minimization properties. |